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ABSTRACT 

C.F. Leyba conducted a longitudinal study (1972-77) 
of the Title VII Bilingual Education Program in the Santa Fe Public 
Schools involving 1,257 students in* grades 1-6. Three groups were 
studied: a longitudinal treatment group (LTG) which received 
uninterrupted bilingual education for a span of years; a 
nonlongitudinal treatment group which received bilingual education 
two or more years but only intermittently, due to dropping out and 
later returning; and control groups which were randomly selected each 
year from Title VII participating schools. The present study 
re-examined the Leyba data for the LTG and controls employing 
meta-analysis to obtain an overall estimate of the effect size of the 
program in reading and mathematics and to test the effect size for 
significance • The study used a weighted, unbiased estimate of effect 
size. Leyba reported a total of 16 comparisons of the LTG and 
controls in reading and 16 comparisons in mathematics. Data were 
analyzed using a computer program. Analysis showed an overall effect 
of the program on the students' mathematics achievement. The overall 
effect of the program did not reach statistical significance in 
reading. Results suggested gains for bilingual education students 
which clearly do not support a hypothesis of deleterious effects of 
bilingual education. (NQA) 
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Leyba (1978) conducted a longitudinal study of the Title VII 
Bilingual Education Program in the Santa Fe Public Schools 1972 - 1977 
involving 1257 students in Grades 1 - 6. This program* was identified as 
one of seven exemplary bilingual education programs by the American 
Institutes of Research (Leyba 1978). A description of the program is 
contained in the Leyba study (Leyba 1978).. 

Students in schools eligible for Title VII funding came from a 
bilingual home and had limited English speaking proficiency. However, 
to be a participant in the bilingual program, parents had to volunteer 
their child for the program. Each year control groups were randomly 
selected at each school. Among other measures, students were pre-tested 
(fall) and post-tested(spring) with the Metropolitan Achievement Test, 
Total Reading and Total Mathematics subtests. 

Leyba studied three groups (1) a longitudinal treatment group (LTG) 
which received uninterrupted bilingual education for a span of years; 
(2) a nonlongitudinal treatment group (NLTG) which received bilingual 
education two or more years but only intermittently, due to dropping out 
and later returning, and (3) control groups which were randomly selected 
each year from Title VII participating schools. Four longitudinal groups 
were identified: Group A consisted of students enrolled in the bilingual 
program for three years beginning with Grade 1; Group B consisted of 
students enrolled in the bilingual program for four years beginning with 
Grade 1; Group C included students enrolled in the bilingual education 
program for five years beginning with Grade 2; and Group D consisted of 
students enrolled in the bilingual program for five years beginning with 
Grade 2. 



Leyba computed the post-test minus pre-test for each student in 
reading and mathematics for the LTG, NLTG, and controls in Groups A - D. 
Independent t- tests were used to compare the mean difference scores of, the 
LTG with the NLTG, the LTG with the controls, and the NLTG with the controls 
Leyba (1978, p. ii) concluded from the comparisons of the NTG, NLTG, and 
controls that "The Title VII students over time in the majority of cases 
outperformed the non-Title VII students in reading and mathematics. 

Several limitations surround Leybas (1978) conclusions: 

1. Only 21% of the reported comparisons of the LTG and NLTG with 
controls yielded statistically significant results favoring LTG or NLTG 
groups. A total of 60% of the comparisons yielded positive mean differences 
which favored bilingual program students. These suggested a positive effect 
of the bilingual program. It is unclear how Leyba (1978) reached his 
conclusions about the favorable impact of the bilingual program with so 

few statistically significant results. 

2. Leyba performed multiple comparisons of the LTG, NLTG, and control 
groups at the .05 level of significance. Miller (1981 ^ has shown that such 
comparisons will change the experiment-wise error rate from .05 to .14 
which will increase the number of untrue significant findings (Type I errors 

The purpose of the present study is to re-examine the Leyba data for 
the LTG and controls employing meta-analysis (1) in order to obtain an 
overall estimate of the effect size of the Santa Fe Bilingual Education 
Program in reading and mathematics and (2) to test the effect size for 
si gni f icance. 
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METHOD 

The effect size has been described by Glass (1976) as the difference 
between the experimental group mean and the control group mean divided 
by the standard deviation of the control group. Hedges (1981) has shown 
that Glass's effect size has a small bias which can be corrected. He 
provides methods for computing an unbiased effect size for a series of 
experiments as well as a z-test for the significance of the effect size. 
Further, Hedges' (1981) procedures allow for pooled variances to be used. 
The present study used a weighted, unbiased estimate of effect size because 
weights give "more weight to experiments which contribute more precise 
information to the overall effect size estimate" (Hedges, 1981, p. 125). 
Since Leyba did not provide stanc ard deviations in his report, effect sizes 
were obtained with the following Tormula: 

ES = t V(l/ryhl/n 2 ) 

Refer to Appendix A for the data of each cohort. 

PROCEDURE 

The present analysis concerned only the LTG and control groups. Because 
the LTG group received uninterrupted bilingual education, it appeared that the 
LTG group would most clearly reflect the impact of the bilingual education 
program. Leyba (1978) reports a total of 16 comparisons of the LTG and 
controls in Heading and 16 comparisons in mathematics. A computer program 
was written (Powers, 1983) to analyze the data. 
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FINDINGS AND DISCUSSION 

Effect sizes for reading ranged from .92 to -.39, and in mathematics 
from 1.27 to -.43. Refer to Appendixes B and C for computational details. 
The overall average effect size in reading was .12 (z = 1.48, p< .14) and - 
for mathematics was .29 (z = 3.63, p<.01). The effect size in reading 
was positive but it did not reach significance whereas in mathematics* 
the effect size was significant beyond the .01 level. Pearson correlations 
between grade level and effect sizes reveal a negative correlation of 
-.52 (p < .05) in reading and -.53 (p < .05) in mathematics.' The negative 
correlation indicates that initial impact was greater in earlier grades. 

The present analysis has shown art overall effect of the Santa Fe 
bilingual Education program on the mathematics achievement of students. The 
overall effect of the bilingual program did not reach statistical significance 
in reading. However, the results suggested gains for bilingual education 
students A clearly do not support a hypothesis of deleterious effects of 
bilingual education. 

r 



9 

ERIC 



6 



- 5 - 



FOOTNOTE 

^he authors thank Darrell L, Sabers, Department of Educational 
Psychology, University of Arizona, for his comments on an earlier version of 
this paper. Copies of this paper may be obtained from Stephen Powers, Ph.P, 
Legal and Research Services, Tucson Unified School District, P. 0. Box 40400, 
Tucson, Arizona 85717. 
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APPENDIX A 

Sample Sizes and T-Values Comparing Bilingual and Control Groups 
as Reported in Leyba (1978) 



Cohort 




Grack- 1 


Grade 


2 


Grade 


3 


Grade 4 




Grade 5 




Grade 6 




N l 


N 2 t 


N l 


N 2 


t 


N l 


N 2 


t 




N 2 


t 




N 2 


t 


N l 




t 


Reading 




































A 


36 


zl .56 a 


' '32 


JO 


.85 


32 


" 19 


.10 




















B 


23 


18 2.97* 


23 


17 


2.21* 


23 


19 


.59 


23 


19 


-.20 














C 












26 


23 


-.12 


26 


22 


-.34 


26 


19 


-1.30 


26 


23 


.75 


D 






16 


9 


-.14 


16 


19 


-.16 


16 


23 


.00 


16 


20 


.18 


,16 


22 


,15 


Mathematics 




















> 
















A 


36 


24 .28 a 


32 


20 


2.82* 


32 


19 


.71 ' 
















• 




B 


23 


18 4.11* 


23 


17 


2.51* 


23 


19 


-.49 


23 


19 


-.39 














C 












26 


23 


.86 


26 


22 - 


1 .50 • 


26 


19 


.38 


26 


23 


-.31 


D 






16 


9 


2.99* 


16 


19 


1 .68 


16 


23 


.11 


16 


20 


1 .58 


16 


22 


.82 


*p < .05 
a These values 


were 


incorrectly 


reported 


in 


Leyba 


(1978) 


as - 


.56 and 


-.28 



















o 

ERIC 



READ I K W 
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SUE ESTIMATOR FOR A SERIES 
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EFFECT SIZE ESTIMATOR FOR A SERIES UF EXPERIMENTS 
NJMBFK UF RXDL'RIMENTS s 16 
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